feat(webhooks): verify_and_parse_* API for compressed payloads (CHA-3071)#211
feat(webhooks): verify_and_parse_* API for compressed payloads (CHA-3071)#211nijeesh-stream wants to merge 7 commits into
Conversation
… (CHA-3071)
Adds two new helpers on `StreamChat::Client` so customers can ingest
gzip-compressed webhook payloads (and base64-wrapped SQS / SNS firehose
envelopes) without wiring `Zlib`, `Base64`, and `OpenSSL::HMAC`
together themselves:
* `decompress_webhook_body(body, content_encoding = nil, payload_encoding = nil)`
- the primitive decoder, no signature check
* `verify_and_decode_webhook(body, x_signature, content_encoding = nil, payload_encoding = nil)`
- decodes and timing-safely verifies the X-Signature HMAC against the
uncompressed JSON, raising `StreamChat::WebhookSignatureError` on
any failure (decode, decompress, or signature mismatch).
`nil` / empty for either encoding is a no-op, so the same handler keeps
working whether or not compression is enabled at the dashboard.
The existing `verify_webhook(body, x_signature)` boolean helper is left
untouched for backward compatibility.
Helper logic lives in `lib/stream-chat/webhook.rb` so the decode
primitives can be exercised independently. New tests in
`spec/webhook_compression_spec.rb` cover plain / gzip / base64 / base64
+ gzip round-trips, case-insensitive encoding values, every documented
unsupported encoding, malformed gzip / base64 input, and signature
mismatch scenarios (including signatures wrongly computed over the
compressed or wrapped bytes).
Docs: new "Compressed webhook bodies" section in the webhooks overview
with a Rails example and an SQS / SNS subsection that passes
`payload_encoding: 'base64'`.
Co-authored-by: Cursor <cursoragent@cursor.com>
Replaces verify_and_decode_webhook / decompress_webhook_body with the cross-SDK contract documented at https://getstream.io/chat/docs/node/webhooks_overview/. Module-level helpers in StreamChat::Webhook: Primitives: ungzip_payload - gzip magic-byte detection + inflate decode_sqs_payload - base64 then ungzip-if-magic decode_sns_payload - alias for decode_sqs_payload verify_signature - constant-time HMAC-SHA256 comparison (parameter order matches the cross-SDK spec) parse_event - JSON -> Hash (typed event lands later) Composite (return parsed event Hash): verify_and_parse_webhook verify_and_parse_sqs verify_and_parse_sns The composite functions auto-detect compression from body bytes, so the same handler stays correct whether or not Stream is currently compressing payloads, and behind middleware that auto-decompresses. Client#verify_and_parse_{webhook,sqs,sns} mirror the three composite helpers with the api_secret pulled from the client. The legacy Client#verify_webhook(body, signature) -> bool helper is kept for backward compatibility (now delegates to verify_signature). Co-authored-by: Cursor <cursoragent@cursor.com>
RFC 1952 defines the gzip magic number as the two-byte sequence 1F 8B; the third byte (CM) is informational and not part of the identifier. Trim the magic check from three bytes to two to match the spec and stay consistent with the reference implementations in the public docs. Co-authored-by: Cursor <cursoragent@cursor.com>
The previous parent (`StreamAPIException`) declares a non-nilable `Faraday::Response` and an HTTP-shaped `initialize`. A local webhook verification failure has no response, so the subclass was forced to skip `super`, fake `@error_code`/`@error_message`/`@json_response`, and call `StandardError#initialize` via `bind_call` to populate the message - while still exposing a `response` reader that violates its own type signature by returning nil. Inherit directly from `StandardError` instead. Drops the LSP violation, the `Lint/MissingSuper` disable, and three irrelevant accessors. No caller depended on the prior parent: the class was introduced in this PR and is only used by the webhook helpers. CHA-3071 Co-authored-by: Cursor <cursoragent@cursor.com>
The Ruby code samples still referenced an earlier draft of the API (`decompress_webhook_body`, `verify_and_decode_webhook`, with `content_encoding` / `payload_encoding` arguments returning raw JSON bytes). The shipped API renamed those to `verify_and_parse_*`, removed the encoding arguments (gzip is detected from the body bytes per RFC 1952), and returns a parsed `Hash` rather than bytes - so the previous snippets would have failed with NoMethodError, and the subsequent `JSON.parse(json_bytes)` would have raised TypeError on the returned Hash. Replace both snippets with the current API, add a pointer to the module-level primitives, and note that the legacy boolean `verify_webhook` remains for callers that don't need compression support. CHA-3071 Co-authored-by: Cursor <cursoragent@cursor.com>
mogita
left a comment
There was a problem hiding this comment.
Cross-SDK review pass for CHA-3071. Two inline comments — see below.
decode_sns_payload now JSON-parses the SNS HTTP notification envelope
({"Type":"Notification","Message":"..."}) and extracts the inner
Message field before running the SQS pipeline. Falls through to the
pre-extracted Message string when the input is not a JSON envelope so
existing call sites keep working.
Spec adds a realistic SNS HTTP notification body fixture and exercises
both the new envelope path and the existing pre-extracted Message path.
Co-authored-by: Cursor <cursoragent@cursor.com>
|
Cross-SDK coordination: unifying webhook exception types After the review pass across all 6 SDKs in this rollout and team discussion, we're consolidating the new webhook exception strategy to a single unified exception class rather than the split (signature vs parse exceptions) being introduced in this PR. The Webhook Handling Spec on Notion (CHA-2961) has been revised to reflect this — §5.2 / §5.3 / §7 now specify a single class. Why unified: From a customer's perspective, all failure modes — signature mismatch, gzip decompression failure, base64 decode failure, SNS envelope failure, JSON parse failure, missing schema field — terminate at the same Class name family: Per-SDK naming across the rollout:
Asks for this PR:
This same comment is being posted on all 6 SDK PRs (JS / Go / Ruby / PHP / Java / .NET) for coordination. Happy to discuss naming or scope tradeoffs. |
…den fixtures (CHA-3071) Per Tommaso's suggestion, align the gzip helper with the GNU `gunzip` command name. The function was added in this PR and not yet released, so this is a straight rename with no back-compat alias. Adds Tommaso's reference fixtures to the test suite as named cases so future SDKs can sanity-check against the same payloads: aGVsbG93b3JsZA== -> helloworld (base64) H4sIAGrYAWoAA8tIzcnJL88vykkBAK0g6/kKAAAA -> helloworld (base64+gzip) Co-authored-by: Cursor <cursoragent@cursor.com>
Summary
Adds first-class support for gzip-compressed webhook payloads (HTTP webhooks, SQS, SNS) and exposes a stable
verify_and_parse_*API that mirrors the cross-SDK contract published in Webhooks Overview.New public API (
StreamChat::Webhook)Module-level primitives:
gunzip_payload(body) -> String— gzip-magic-byte detection, no-op when not compresseddecode_sqs_payload(body)— base64 decode then gunzip-if-magicdecode_sns_payload(notification_body)— JSON-parse the SNS HTTP notification envelope, extract the innerMessage, then run the SQS pipeline. Falls through to a pre-extractedMessagestring when the input is not a JSON envelopeverify_signature(body, signature, secret) -> bool— HMAC-SHA256 over the uncompressed body, with a constant-time comparison (matters for the HTTP webhook path where theX-Signatureheader is exposed publicly; SQS / SNS deliveries arrive over AWS-internal transports where timing-attack resistance is not strictly required)parse_event(payload) -> Hash— JSON →HashModule-level composites (return
Hash):verify_and_parse_webhook(body, signature, secret) -> Hashverify_and_parse_sqs(body, signature, secret) -> Hashverify_and_parse_sns(body, signature, secret) -> HashThe
StreamChat::Clientinstance also exposesverify_and_parse_webhook/verify_and_parse_sqs/verify_and_parse_snsthat use the configured@api_secret.Backwards compatibility
Client#verify_webhookis preserved and now delegates toStreamChat::Webhook.verify_signature. The experimentaldecompress_webhook_bodyandverify_and_decode_webhooksurfaces are removed (they were never released).Tests
spec/webhook_compression_spec.rbcovers plain / gzip / base64 / base64+gzip payloads, signature mismatches, malformed bytes, and JSON parsing intoHash. Linked Linear ticket: CHA-3071.Golden test fixtures (Tommaso)
Added shared reference fixtures to the test suite so future SDKs can sanity-check decoders against the same payloads:
Test plan
bundle exec rspec spec/webhook_compression_spec.rb— 28 examples, 0 failuresbundle exec rubocop— clean on touched files